Skip to content

[Other] setup: the SYCL setup on an integrated Arc (xe): native launcher, a RAM fit rule, no KV streaming - #2115

Open
andersson82 wants to merge 2 commits into
Niko1221:mainfrom
andersson82:sycl-igpu-setup
Open

[Other] setup: the SYCL setup on an integrated Arc (xe): native launcher, a RAM fit rule, no KV streaming#2115
andersson82 wants to merge 2 commits into
Niko1221:mainfrom
andersson82:sycl-igpu-setup

Conversation

@andersson82

@andersson82 andersson82 commented Oct 12, 2026 •
edited
Loading

Copy link
Copy Markdown

Problem and result

./setup.sh --backend sycl on a PC whose only Intel GPU is integrated (an Intel GPU at PCI 00:02.0 on xe) stops
with "no Intel Arc found". With this change it finds the GPU, checks that oneAPI and the engine build are on the PC,
offers the sizes that fit the RAM, and writes a config for sycl/serve/strata-sycl-native.sh (no Docker). On the Arc
140T with Strata 0.2.2 that config serves 10 of 10 requests (8.4-12.7 tok/s decode).

Depends on #2114 (the launcher and the iGPU memory query): until that one is merged this diff also shows its commit;
the change of this pull request is the last commit.

Related discussion

None agreed in an issue; it is the setup half of #2114 (platform support for an integrated Arc, which the maintainers
have no machine to gate). Rebased on 0.2.2 and re-run; rebased again on 0.2.3 on 2026-10-23 (one conflict, see the pictures bullet below).

Approach

sycl/setup_intel.py:

  • intel_gpus lists the integrated GPU (on xe) with setup's unified-memory fields; i915_igpus finds one on i915,
    for which setup stops and names the kernel command line that moves it to xe.
  • sycl_engine(igpu=True) also needs /opt/intel/oneapi/setvars.sh (or ONEAPI_ROOT) and says how to build natively.
  • to_sycl_igpu: the PC's paths, the launcher as exe, --stream-experts (written once, also when a hand-made config
    already had it), no --kv-resident (the KV is in the same RAM either way), the reserve as on a card (1,024 MiB up to
    32K, 2,048 above; a given one is kept), SYCL_CACHE_PERSISTENT=1 on the 140T only (it crashed on Xe2),
    STRATA_SYCL_BIN for an AOT build.
  • New since the first version, found in today's run on 0.2.2: setup's new --ple-io ram rule read the 1024 GB stand-in
    RAM the Intel path gives setup and wrote --ple-io ram, which on a 62 GB iGPU would put the 29 GB n-gram table in
    the GPU's own memory beside the 36.5 GiB expert file. to_sycl_igpu leaves it out (one in the user's user_args is
    kept) and the "1024 GB of RAM" line is replaced by a plain one. Also: in --check, Gyro sizes say they are not tested
    on an integrated GPU, and pictures stay off on an integrated GPU with a line that says why (see the next bullet).
  • New with the rebase on 0.2.3: main now has pictures on for every Arc and prints the B-series line for any xe id that
    is not an Alchemist one (is_b_series), which includes the 140T (7d51). On an integrated GPU setup instead keeps
    pictures off unless --vision is given ("nobody has run them on one yet (--vision yes turns them on)") and does not
    print the B-series line. A card takes main's path unchanged. Main's test that no --vision none is left in
    setup_intel.py now checks that the one left is in the integrated-GPU branch only.
  • The fit rule: shard 1 + 20 GiB <= RAM (measured on 0.1.40: 10-13 GiB left with a 36.5 GiB shard 1, 3-4 GiB left with
    43.8 GiB did not run well). The size menu marks the sizes that need more and recommends the first that fits; a size
    that does not fit is asked about (--model X --yes installs it anyway); --check says the same per model.
  • setup's AMD wording (BIOS carve-out, "this AMD card") is rewritten for an iGPU.

tools/test_setup_sycl.py: 8 new tests (detection on xe and i915, the config, the fit rule, the persistent cache, the
launcher's binary path, --ple-io, --stream-experts once). docs/INTEL.md: setup's bullet and a "With setup"
paragraph in "Integrated Arc".

Compatibility: discrete cards are unchanged. Every new branch needs the GPU at 00:02.0, where a card behind a PCIe port
never is. setup.py is not touched.

Validation

Revision for the tests: main 5c8dd32 (v0.2.3) + #2114 + this commit (2026-10-23). The runs on the 140T are from 2026-10-22 on
main 011c9e6 (v0.2.2) + #2114 + this commit. Hardware: Arc 140T (8086:7d51), Core Ultra 9 285H, 62 GiB RAM,
Fedora 44 (kernel 7.2.11, xe), oneAPI 2026.0, Python 3.14.

Passed:

Failed:

  • tools/test_setup_golden.py (48 failures) and tools/test_setup_unsloth.py (4 failures) fail with the same counts on
    main 5c8dd32 without this branch (on 011c9e6 they were CUDA multi-GPU expectations, for example cfg["gpu"] [0, 1]
    vs [1, 0]).
  • tools/test_setup_download.py is flaky on main too. On v0.2.3 it failed 5 of 6 runs on main 5c8dd32 and 5 of 8 on this
    branch (test_a_killed_download_continues and the two NoRangeSupport tests; a SYCL build ran on the same PC during
    most of these runs). This branch does not touch the download code.

Not run:

  • The real ./setup.sh run and the served requests on 0.2.3: they are from 0.2.2 (above).
  • Pictures on the 140T.
  • A discrete Intel card with this branch (I have none); the unit tests for the card path pass.
  • A PC with a discrete card and an iGPU together (setup should pick the card; unit-tested only).
  • Another integrated GPU than the 140T, an iGPU on i915 on real hardware (the message is unit-tested only), Windows.
  • The interactive size menu by hand (unit-tested; the real run used --model IQ1_M --yes).

Limits

  • From reading the code only (unconfirmed static analysis, not run): the same stand-in RAM reaches setup's --ple-io ram
    rule on a discrete Intel card too, and to_sycl keeps what setup wrote. That is a separate problem from this pull
    request and I did not change it.
  • setup's hybrid-CPU rule writes --pool-workers 10 here, and the engine warns the process can run only 10 CPU threads
    ("Recommended: --pool-workers 9"). It made no measurable difference, so it is left as setup writes it.

jielin99 added a commit to jielin99/Strata-codex that referenced this pull request Oct 13, 2026
@jielin99 @claude
tools: accept the NPU plugin's string EXECUTION_DEVICES; record lapto…
e765c76 
…p reports

The NPU plugin returns EXECUTION_DEVICES as the plain string "NPU" (CPU and
GPU return a list), so the probe split it into ['N', 'P', 'U'] and rejected a
valid placement. Wrap a string in a list first; a CPU fallback or an empty
list still fails. Rerun on the laptop: the 64x64 FP32 MatMul passes on NPU
(max abs error 4.9e-05). This only shows the NPU runs a small graph; it is not
Strata on the NPU.

The handoff doc gains the Ubuntu dev machine inventory (no NPU, 15 GiB RAM:
no real model baseline possible there), the review of upstream PRs Niko1221#2079,
Niko1221#2110, Niko1221#2114 and Niko1221#2115 against this branch, and the laptop's hardware and
OpenVINO reports (Core Ultra 9 285H, Arc 140T, Intel AI Boost, 63.5 GiB).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
jielin99 added a commit to jielin99/Strata-codex that referenced this pull request Oct 13, 2026
@jielin99 @claude
tools: add a SYCL capability probe for the Windows laptop
7a67c47 
probe_sycl.cpp reports, per SYCL device, what the native Windows engine
(PR Niko1221#2079) gates on - ext_intel_free_memory and ext_oneapi_graph - plus
host_unified_memory, memory sizes, the largest allocation, sub-group sizes
and fp16/fp64, and runs one 32-wide sub-group kernel per GPU. Read-only, no
large allocation. PROBE_SYCL.bat builds it with icx and runs it three ways
(no switches, ZES_ENABLE_SYSMAN=1, plus UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS=1).

Built with icpx 2025.3.2 (intel/oneapi-basekit image) and run on a UHD 730
over Level Zero: valid JSON, sub-group kernel ok in all three. There the iGPU
reports ~270 MiB free with ~13 GiB MemAvailable, and no ext_oneapi_graph.
The Windows build and the Arc 140T are not tested.

The handoff doc records the laptop toolchain, the cherry-picked upstream PRs
(Niko1221#2114, Niko1221#2110, Niko1221#2079; Niko1221#2115 left out), the setup test results and the probe run.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Niko1221

Niko1221 commented Oct 19, 2026

Copy link
Copy Markdown
Owner

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

  1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
  2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
  3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
  4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.

@Niko1221 Niko1221 closed this Oct 19, 2026
@andersson82 andersson82 mentioned this pull request Oct 22, 2026
Open
@andersson82 andersson82 changed the title setup: the SYCL setup on an integrated Arc (xe): native launcher, a RAM fit rule, no KV streaming [Other] setup: the SYCL setup on an integrated Arc (xe): native launcher, a RAM fit rule, no KV streaming Oct 22, 2026
@andersson82

andersson82 commented Oct 22, 2026

Copy link
Copy Markdown
Author

/reopen

@github-actions github-actions Bot reopened this Oct 22, 2026
@github-actions

github-actions Bot commented Oct 22, 2026

Copy link
Copy Markdown

Reopened, thank you. A maintainer will pick it up from here. If it conflicts with main, rebase it first; that speeds up review.

It has conflicts with main now: please rebase it.

andersson82 and others added 2 commits October 23, 2026 12:06
@andersson82 @claude
sycl: run on an integrated Arc (Arc 140T, xe): free memory from /proc…
9ca7635 
…/meminfo, no prompt-slot borrowing, a native launcher

On the Arc 140T (Core Ultra 9 285H, 62 GiB, Fedora 44, xe, oneAPI 2026.0 on the PC) the engine did not start: the
device has no ext_intel_free_memory (also with ZES_ENABLE_SYSMAN=1), so the expert cache was sized from 0.

- sycl/include/dpct/device.hpp: get_memory_info on a device with host_unified_memory and no free-memory query
  reads MemAvailable from /proc/meminfo (Linux); STRATA_SYCL_FREE_MIB sets it by hand. Other devices: unchanged.
- sycl/src/program/generate.cpp: no prompt-slot borrowing on a host_unified_memory device. With it the answer
  after a borrowing prompt was token 0 (expert activations NaN from the 2nd chunk; 0.1.40, any context).
- sycl/serve/strata-sycl-native.sh (new): the config's exe for the engine on the PC's own oneAPI, no Docker.
- docs/INTEL.md: "Integrated Arc: the Arc 140T"; docs/INTEL_ARC.md: the 140T is no longer "not tested".

Measured on the 140T, Coder IQ1_M, engine 0.1.43 with Niko1221#2110, through the server: 12.0-17.4 tok/s decode on
100-token answers (all 12,288 experts cached), a 72,648-token prompt read at 156.5 tok/s with its first-line
fact recalled. Built with sycl/tools/build.sh (native oneAPI): 0 errors. No discrete card was run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@andersson82
setup: the SYCL setup on an integrated Arc (xe): native launcher, a R…
9f06dd4 
…AM fit rule, no KV streaming

On an integrated GPU setup also leaves out the --ple-io ram its RAM rule chose from the 1024 GB stand-in (a user's
own is kept), writes --stream-experts once when the config already had it, says Gyro sizes are not tested there in
--check, and names the integrated GPU in the pictures-off line.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@andersson82 @Niko1221